Repository navigation
feat: Support cos_sin_cache in all cases. - #3517
Conversation
|
/bot run --disable-fail-fast |
|
PR_Github #2109 [ run ] triggered by Bot |
|
PR_Github #2109 [ run ] completed with state |
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
|
/bot run --disable-fail-fast |
|
PR_Github #2291 [ run ] triggered by Bot |
|
PR_Github #2291 [ run ] completed with state |
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
|
/bot run --disable-fail-fast |
|
PR_Github #2332 [ run ] triggered by Bot |
|
PR_Github #2332 [ run ] completed with state |
|
/bot reuse-pipeline |
|
PR_Github #2422 [ reuse-pipeline ] triggered by Bot |
|
PR_Github #2422 [ reuse-pipeline ] completed with state |
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
This MR contains the following updates:
fuse_pos_embd=True/Falseand createRotaryEmbeddinginside attention module, so that the users don't need to handle it in the modeling files.cos_sinfor unfused rope implementation. If flashinfer is available, useapply_rope_with_cos_sin_cache_inplaceinstead ofapply_rope_inplace. Otherwise, we fallback to pure pytorch implementation, which can support any rope now.